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REMARKS 

Claims 1, 5-13, 21, and 24-26 are pending, with claims 1, 10, 13, 21, and 26 being 
independent. Claims 1, 5, 6, 10, 13, 21, and 26 have been amended and claims 2-4, 14-20, 22, 
23, and 27-31 have been canceled. Support for the amendments can be found in the originally- 
filed specification, at least at page 2, line 8 to page 4, line 6. No new matter has been introduced. 

Independent claim 1 recites a method of recognizing punctuation in computer- 
implemented speech recognition. The method includes performing speech recognition on an 
utterance to produce a recognition result for the utterance, identifying a non-verbalized 
punctuation mark in a recognition result including predicting the non- verbalized punctuation 
mark using at least one text feature and at least one acoustic feature related to the utterance, 
inserting the non- verbalized punctuation mark into the recognition result, and formatting the 
recognition result based on the identification of the non- verbalized punctuation mark after the 
non- verbalized punctuation mark has been inserted in the recognition result. The acoustic 
feature includes one or more of a length of a period of silence and a function of pitch of words 
near the period of silence. 

Independent claim 13 recites an apparatus including a computer-readable medium having 
instructions stored thereon that when executed by a machine result in at least the following steps. 
Speech recognition on an utterance is performed to produce a recognition result for the utterance. 
A non-verbalized punctuation mark is identified in a recognition result by predicting the non- 
verbalized punctuation mark using at least one text feature and at least one acoustic feature 
related to the utterance. The non-verbalized punctuation mark is inserted into the recognition 
result, and the recognition result is formatted based on the identification of the non- verbalized 
punctuation mark after the non-verbalized punctuation mark has been inserted into the 
recognition result. The acoustic feature includes one or more of a length of a period of silence 
and a function of pitch of words near the period of silence. 

Independent claim 21 recites a method of recognizing punctuation in computer- 
implemented speech recognition dictation. The method includes performing speech recognition 
on an utterance to produce a recognition result for the utterance, identifying a non-verbalized 
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punctuation mark in a recognition result, determining where to insert the non-verbalized 
punctuation mark within the recognition result based on the identification using at least one text 
feature and at least one acoustic feature related to the utterance to predict where to insert the non- 
verbalized punctuation mark, and inserting the non-verbalized punctuation mark into the 
recognition result. The acoustic feature includes one or more of a length of a period of silence 
and a function of pitch of words near the period of silence. 

Independent claim 26 recites an apparatus including a computer-readable medium having 
instructions stored thereon that when executed by a machine result in at least the following steps. 
Speech recognition is performed on an utterance to produce a recognition resuh for the utterance. 
A non-verbalized punctuation mark is identified in a recognition result. It is determined where to 
insert the non- verbalized punctuation mark within the recognition resuh based on the 
identification using at least one text feature and at least one acoustic feature related to the 
utterance to predict where to insert the non- verbalized punctuation mark, and the non- verbalized 
punctuation mark is inserted into the recognition result. The acoustic feature includes one or 
more of a length of a period of silence and a fiinction of pitch of words near the period of silence. 

Claims 1, 13,21, and 26 have been rejected as being anticipated by U.S. Patent No. 
6,778,958 (Nishimura). Applicant requests withdrawal of this rejection because Nishimura fails 
to describe or suggest formatting a recognition result based on an identification of a non- 
verbalized punctuation mark after the non-verbalized punctuation mark has been inserted into the 
recognition result, where the non-verbalized punctuation mark is predicted using at least one text 
feature and at least one acoustic feature that includes one or more of a length of period silence 
and a function of pitch of words near the period of silence, as recited in independent claims 1 and 
13, and inserting a non- verbalized punctuation mark into a recognition result, where the non- 
verbalized punctuation mark is identified using at least one acoustic feature that includes one or 
more of a length of a period of silence and a function of pitch of words near the period of silence, 
as recited in independent claims 21 and 26. 

Nishimura relates to a speech recognition apparatus that uses a general-purpose language 
model and a punctuation mark language model that includes only words that frequently appear 
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before and after a punctuation mark to identify an unvoiced, pause location for the insertion of a 
punctuation mark such as a comma. See Nishimura at Abstract and col. 4, lines 22-39. The 
punctuation mark language model detects a blank portion such as a pause in acoustic model data 
and regards a punctuation mark as a word and automatically inserts a punctuation mark in the 
blank portion identified as a pause. See Nishimura at col. 5, lines 5-63. 

However, Nishimura never describes or suggests that the acoustic model data used to 
detect the blank portion include either a length of a period of silence or a function of pitch of 
words near the period of silence. Rather, the acoustic model data merely provides the blank 
portion or pause in speech data. See Nishimura at col. 5, lines 10-33. 

Moreover Nishimura's apparatus also does not format the format a recognition result 
based on an identification of the punctuation mark after the punctuation mark has been inserted 
into the recognition resuh. Nishimura merely explains that the punctuation mark is automatically 
inserted into the text data. See Nishimura at col. 4, lines 50-62. 

Accordingly, claims 1, 13, 21, and 26 are allowable over Nishimura. 

Claims 5, 6, 24, and 25 have been rejected as being unpatentable over Nishimura in view 
of U.S. Patent No. 7,089,184 (Rorex). Applicant requests withdrawal of this rejection for the 
following reasons. Claims 5, 6, 24, and 25 depend from independent claims 1 or 21, which were 
rejected as being anticipated by Nishimura. As discussed above, Nishimura fails to describe or 
suggest formatting a recognition result based on an identification of a non- verbalized punctuation 
mark after the non- verbalized punctuation mark has been inserted into the recognition result, 
where the non-verbalized punctuation mark is predicted using at least one text feature and at 
least one acoustic feature that includes one or more of a length of period silence and a function of 
pitch of words near the period of silence, as recited in independent claim 1, and inserting a non- 
verbalized punctuation mark into a recognition result, where the non-verbalized punctuation 
mark is identified using at least one acoustic feature that includes one or more of a length of a 



Applicant has omitted claim 26 from the recitation of rejected claims foimd in section 8 of the office action 
because it appears that claim 26 was listed in error since it is not discussed in detail in section 8. Claim 26 is 
discussed in detail at other sections of this Reply. 
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period of silence and a function of pitch of words near the period of silence, as recited in 
independent claim 21 . 

Moreover, Rorex does not remedy the failure of Nishimura to describe or suggest 
formatting a recognition resuh based on an identification of a non-verbahzed punctuation mark 
after the non-verbalized punctuation mark has been inserted into the recognition result, and it 
would not have been obvious to modify Nishimura to identify the non- verbalized punctuation 
mark using at least one acoustic feature that includes one or more of a length of a period of 
silence and a function of pitch of words near the period of silence based on the teachings of 
Rorex. 

First, Rorex never describes or suggests that a recognition result is formatted based on an 
identification of a non-verbalized punctuation mark. Rorex merely explains at col. 10, lines 19- 
22 that "predetermined speech features may be used to add punctuation . . . ." Indeed, Rorex 
provides no additional information about what happens after punctuation is added. 

Second, although Rorex mentions that tonality rise may be used to add punctuation such 
as a question mark (see Rorex at col. 10, lines 24-27), Rorex never mentions that a length of a 
period of silence is used to identify a non-verbalized punctuation mark. 

Third, it would not have been obvious to modify Nishimura to use acoustic features such 
as tonality as described in Rorex because such a modification of Nishimura would change the 
principle of operation of Nishimiira. Nishimura's invention relates to using the restrictive 
punctuation mark language model that includes only words that appear before and after a 
punctuation mark to determine whether to insert a punctuation mark. Thus, modification of 
Nishimura's apparatus to include acoustic features such as length of a period of silence or pitch 
would render Nishimura's restrictive punctuation mark language model unnecessary. 
Additionally, modification of Nishimura to include these other acoustic features would require a 
substantial redesign of the algorithm used in Nishimura's speech recognition apparatus to 
account for these acoustic features because Nishimura's speech recognition apparatus is set up to 
detect only the pause. 
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Accordingly, claims 1 and 21 are allowable over any proper combination of Nishimura 
and Rorex. Dependent claims 5, 6, 24, and 25 are allowable for at least the reasons that claims 1 
and 21 are allowable and for containing allowable subject matter in their own right. For 
example, claims 6 and 25 recite that the acoustic feature includes a ratio of an average pitch of 
words near the period of silence. However, neither Nishimura nor Rorex describes or suggests a 
ratio of average pitch of words near a period of silence. Rorex merely mentions that a "tonality 
rise by the speaker may signify a question or query. . . ." See Rorex at col. 10, lines 24-26. Rorex 
never explains how such a tonality rise is measured. 

Claims 7-9 have been rejected as being unpatentable over Nishimura in view of U.S. 
Patent No. 5,920,877 (Kolster). Claims 7-9 depend from claim 1, which was rejected as being 
anticipated by Nishimura. As discussed above, Nishimura fails to describe or suggest formatting 
a recognition resuh based on an identification of a non- verbalized punctuation mark after the 
non- verbalized punctuation mark has been inserted into the recognition result, where the non- 
verbalized punctuation mark is predicted using at least one text feature and at least one acoustic 
feature that includes one or more of a length of period silence and a function of pitch of words 
near the period of silence, as recited in independent claim 1 . 

First, Kolster, which does not describe prediction of non-verbalized punctuation marks, 
also necessarily fails to describe or suggest using an acoustic feature that includes one or more of 
a length of period of silence and a function of pitch of words near the period of silence to predict 
non-verbalized punctuation marks. 

Second, it would not have been obvious to modify Nishimura to include formatting 
described by Kolster because although Kolster mentions that text can be formatted and 
Nishimura predicts punctuation marks in speech, the combination of formatting recognized text 
after insertion of a non-verbalized punctuation mark has not been known and provides a new 
function in speech recognition, which strives to reduce the amount of active work on the part of 
the speaker. Indeed, Nishimura never suggests that such formatting of the text should be 
performed after the punctuation mark is inserted. 
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Accordingly, claim 1 is allowable over any proper combination of Nishimura and 
Kolster, as are dependent claims 7-9. 

Independent claim 10 recites a method of recognizing punctuation in computer- 
implemented speech recognition. The method includes performing speech recognition on an 
utterance to produce a recognition result for the utterance, identifying a non-verbalized 
punctuation mark in a recognition result, formatting the recognition result based on the 
identification, selecting a portion of the recognition result to be corrected that includes the non- 
verbalized punctuation mark, and correcting the portion of the recognition result that includes the 
non-verbalized punctuation mark with one of a number of correction choices. 

Claims 10-13 have been rejected as being unpatentable over Nishimura in view of U.S. 
Patent No. 6,356,865 (Franz). Applicant requests withdrawal of this rejection because, as the 
Office agrees, Nishimura fails to describe or suggest selecting a portion of a recognition result to 
be corrected that includes a non- verbalized punctuation mark that was identified in a recognition 
result and correcting the portion of the recognition result that includes the non-verbalized 
punctuation mark, as recited in independent claim 10, and because it would not have been 
obvious to modify Nishimura in the manner suggested by the Office using the teachings of 
Franz. 

Franz relates to a spoken language translation apparatus that receives speech input such 
as words, sentences, and phrases in a natural spoken language and permits a speaker to select 
alternative utterance hypotheses from a display 1306 of utterance hypotheses. See Franz at col. 
16, lines 22-42 and line 62 to col. 17, line 34 and Fig. 13. However, Franz never suggests that 
punctuations listed in the utterance hypotheses were non-verbalized and therefore identified 
using the apparatus. Moreover, Nishimura never suggests that the recognition result that 
includes the non-verbalized punctuation mark is corrected. Modification of Nishimura to include 
such correction would require a substantial redesign of Nishimura's apparatus to add an 
algorithm for correction to the current algorithm in Nishimura's apparatus. 

Accordingly, claim 10 is allowable over any proper combination of Nishimura and Franz. 
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In conclusion, applicant submits that all claims are in condition for allowance. 

The fee in the amount of $1050 in payment of the three-month extension of time fee is 
being paid concurrently herewith on the Electronic Filing System (EFS) by way of Deposit 
Account authorization. Please apply any other charges or credits to Deposit Account 
No. 06-1050. 

Respectfully submitted, 
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